The present invention relates to a method of calibrating a sensor system, in particular an image processing system.
It is known that image recording or camera systems and other sensor systems are used as a module of a vehicle safety system in road traffic. In this case, information regarding the distance and relative velocity of the vehicle is constantly being processed in relation to other objects, i.e., other vehicles, and the prevailing road conditions. The image recording systems and optionally also radar sensors are used to measure geometric quantities in the vehicle environment; e.g., radar sensors are known from German Published Patent Application No. 42 42 700 A1, for example. With this known arrangement, a cruise control, for example, in a vehicle can be expanded so that driving speed is adapted to slower vehicles in front if the sensor systems detect such vehicles in the anticipated path of the vehicle. For example, the path of the vehicle can be determined with the help of yaw sensors, steering angle sensors or transverse acceleration sensors, on the basis of wheel speeds or also with the above-mentioned image processing systems or navigation systems.
For satisfactory functioning of the system, prior calibration of the sensor system or the individual sensors in the vehicle environment is done in principle for a subsequent accurate measurement of geometric quantities such as lengths, speeds, etc.; this is done in a laboratory-like environment either before or after installation of the sensors in the vehicle. The various calibration methods, usually require controlled movement of the sensors or the objects detected by the sensor systems. Often it is even desirable to observe a specific artificial object, which is then referred to as a calibration field accordingly. To guarantee permanent functional reliability, subsequent repeated checking of the calibration for possible changes is desirable, which may be very complicated.
It is known from European Patent No. 602 013 that ambient information available from a road map, for example, is used to calibrate a compound navigation system. In this case, information regarding the driver""s own vehicle status is processed in combination with known information about the environment available from the map.
A method of calibrating a sensor system which is used to detect and analyze objects in the path of a vehicle is carried out according to the present invention in such a way that characteristic data on the objects,, including the roadway, is detected with the sensor system in an advantageous manner, and data recognized as stationary or quasi-stationary objects, taking into account the movement of the vehicle, is sent to a calibration unit. The deviation in the data measured instantaneously from data provided by a model of the objects is determined as an error vector in this calibration unit and used to correct the data of the model for the purpose of minimizing the deviation, e.g., by an iterative method.
An automatic calibration of a sensor system can be performed in an especially advantageous manner according to the present invention, which also permits subsequent automatic checking of a calibration that has already been performed. To do so, no other equipment is necessary except for the essentially known sensor elements in the vehicle environment and an analyzer unit or calibration unit.
Furthermore, no special movements of the vehicle which previously had to be executed separately for the purpose of calibration are necessary here because the calibration according to the present invention instead utilizes the vehicle""s own movement in operation as well as the knowledge that objects in the three-dimensional world are often rigid and that they move as a whole, and the calibration of the sensors is stable over a longer period of time. In this case, the calibration of the sensor system is quasi-permanent, i.e., it is variable but only gradually. Heuristic assumptions regarding the typical behavior of other vehicles (e.g., remaining in their lane) which can result in malfunctions in practical applications are not necessary for this method. Thus, one property of the present invention is that it permits a calibration which detects the measurable quantities as well as possible. The demand for a so-called true calibration, such as that in the case of a laboratory calibration, is only secondary here.
In addition, from the standpoint of manufacturing technology, installation of uncalibrated sensors is especially advantageous, because the calibration of a sensor is often sensitive to mechanical or thermal stress. Thus, not only is manufacturing simplified, but also it is readily possible to later take into account the effects of the installation or any other changes in the sensor system or the vehicle on the calibration. In addition, the calibration can be checked at any time.
According to an especially advantageous embodiment of the method according to the present invention, there is a first determination of object data which is stored as model data in an initialization phase with preselectable parameters. In all subsequent measurements which are performed on a cyclic basis, the respective instantaneous object data is processed in the calibration unit with the previously determined and stored model data to obtain the respective error vector.
During the processing of data in the calibration unit, the recurring object data from previous measurements is selected, deleting object data not found again and including object data newly added. The object data showing a reduction in the respective confidence interval after repeated measurements from different positions of the vehicle is characterized as data belonging to a stationary or quasi-stationary object.
In the case of the method according to the present invention, a corresponding relative velocity of objects can also be determined from successive object data and then used to determine the vehicle""s own movement. Object data attributable to an object having this same relative velocity can be characterized as data belonging to a stationary or quasi-stationary object. A rotational motion of the vehicle due to pitching and/or cornering can also be used as the vehicle""s own motion in an advantageous manner.
The results of calibration of one sensor of the sensor system can be easily transferred to one or more other sensors on the motor vehicle for calibration of these sensors as well. Thus, with the method according to the present invention, joint calibration of most of the vehicle sensors after installation is possible without requiring any special boundary conditions. In particular, the combination of sensors such as camera, radar, wheel sensor, acceleration sensor is advantageous here, but an application in conjunction with LIDAR sensors or ultrasonic sensors is also possible.
For the case when the sensor(s) of the sensor system yield(s) contradictory measurement data, a signal may be transmitted to an analyzer unit or to the driver of the vehicle, for example.
The present invention can thus be used to advantage to perform a calibration with good results without requiring any special calibration arrangement or a calibration motion sequence. The sensor system or analyzer system may be completely or largely automatically self-calibrating and it also may determine the accuracy and reliability of its own calibration. Parameters that are not or have not yet been calibrated are characterized by an infinite inaccuracy. In particular, the method according to the present invention notices major changes in calibration which are possible due to thermal or mechanical shifts, for example.
The present invention can be used to particular advantage when the sensor system has an image recording system where pixels are detected and analyzed serially by an electronic camera having a nonlinear transformer characteristic in the recording intervals. Many traditional image recording methods perform the camera calibration by the indirect route of fundamental matrices and therefore rely on simultaneous recording of the image elements (cubic image grid) and synchronous sensor data. The method according to the present invention, however, also operates with the individual pixels recorded at any desired point in time and with asynchronous sensor data. Thus, the method according to the present invention permits in particular calibration of high-resolution video cameras having a nonlinear characteristic which usually record their pixels serially.